Designing Zero-Copy FTP Mechanisms to Achieve High Performance Data-Transfer over InfiniBand WAN

FTP has been the most popular method to transfer large files for data-staging, replication, and the like. While existing FTP mechanisms have improved gradually with newer networks, they still inherit the fundamental limitations imposed by the underlying networking protocols (TCP/UDP) they use. These include limited network bandwidth utilization, high memory bandwidth and CPU utilization that TCP/UDP cause on the end-nodes. Thus both the performance and scalability of such systems is limited. The advent of InfiniBand (IB) WAN has enabled the use of high performance transport protocols in the WAN scenarios, which can be leveraged for designing FTP mechanisms. Enabling IB-based FTP capabilities and providing good efficiency for such transfers presents considerable challenge. In this paper we present an Advanced Data Transfer Service (ADTS) to enable efficient data transfers over WAN. We leverage the ADTS’s capabilities to design high performance file transfer mechanisms (FTP based on ADTS). Our ADTS layer improves data transfer performance by optimizing several aspects including efficient buffer management, memory registration cache, pipelining of data transfers, reducing TCP/IP related data copies, and maintaining persistent FTP data sessions. Further, we reduce the CPU utilization required for the data-transfers (by up to a factor of 6) and demonstrate a significantly higher FTP server scalability. In our experimental results, we observe that our FTP-ADTS design outperforms existing TCP and UDP based approaches by more that 80% in transferring large volumes of data. In addition, we utilize the WAN emulation capabilities of Obsidian InfiniBand WAN routers to study the impact of our designs in a wide range of WAN scenarios, leading to solutions that enable the design of highly capable WAN communication protocols required to power the next-generation high performance parallel and distributed environments.